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Abstract. In probabilistic game structures, probabilistic alternating 
simulation (PA-simulation) relations preserve formulas defined in prob- 
abilistic alternating-time temporal logic with respect to the behaviour 
of a subset of players. We propose a partition based algorithm for com- 
puting the largest PA-simulation. It is to our knowledge the first such 
algorithm that works in polynomial time. Our solution extends the gen- 
eralised coarsest partition problem (GCPP) to a game-based setting with 
mixed strategies. The algorithm has higher complexities than those in 
the literature for non-probabilistic simulation and probabilistic simula- 
tion without mixed actions, but slightly improves the existing result for 
computing probabilistic simulation with respect to mixed actions. 

1 Introduction 

Simulation and bisimulation relations are useful tools in the verification of finite 
and infinite state systems. State space minimisation modulo these relations is a 
valuable technique to fight the state explosion problem in model checking, since 
bisimulation preserves properties formulated in logics like CTL and CTL* [9] 
while simulation preserves the universal (or safe) fragment of these logics [15]. 

In some situations, however, it is necessary to model quantitative aspects 
of a system. It is the case, for instance, in wireless networks, where we often 
need to assume that there is a chance of connection failure with a given rate. 
This requires modelling network systems with randomised behaviours (e.g., by 
pooling a connection after uncertain amount of time to minimise conflict) . An- 
other important fact of real-world systems is that environment changes, such as 
unexpected power-off, are often unpredictable. Therefore, we need to encode ap- 
propriate system behaviours to handle such situations, and in order to do so, it is 
sometimes crucial to employ probabilistic strategies to achieve the best possible 
outcomes [28] . One simple example is the rock-scissor-paper game where there is 
no deterministic strategy to win since the other player's move is unknown, but 
there is a probabilistic strategy, sometimes called mixed strategy, to win at least 
a third of all cases in a row, regardless of what the other player does. 3 

A probabilistic game structure (PGS) is a model that has probabilistic tran- 
sitions, and allows the consideration of probabilistic choices of players. The 

3 A mixed strategy also ensures an eventual win but deterministic strategies do not. 



simulation relation in PGSs, called probabilistic alternating simulation (PA- 
simulation), has been shown to preserve a fragment of probabilistic alternating- 
time temporal logic (PATL) under mixed strategies, which is used in character- 
ising what a group of players can enforce in such systems [29] . In this paper we 
propose a polynomial-time algorithm for computing the largest PA-simulation, 
which is, to the best of our knowledge, the first algorithm for computing a simu- 
lation relation in probabilistic concurrent games. A PGS combines the modelling 
of probabilistic transitions from probabilistic automata (PA), and the user inter- 
actions from concurrent game structures (GS). In PA, the probabilistic notions 
of simulation preserve PCTL safety formulas [23]. The alternating simulation [3] 
in GS has been been proved to preserve a fragment of ATL*, under the se- 
mantics of deterministic strategies. These simulation relations are computable 
in polynomial time for finite systems [30, 3]. 

Related work. Efficient algorithms have been proposed for computing the largest 
simulation (e.g., see [16, 26, 5, 14, 27]) in finite systems, with a variety of time and 
space complexities. In particular, Gentilini et al. [14] develop an efficient algo- 
rithm with an improved time complexity based on the work of Henzinger et al. [16] 
without losing the optimal space complexity. Van Glabbeek and Ploeger [27] later 
find a flaw in [14] and propose a non-trivial fix. The best algorithm for simulation 
in terms of time complexity is [20] . To compute probabilistic simulation, Baier et 
al. [4] reduce the problem of establishing a weight function for the lifted relation 
to a maximal flow problem [1]. Cattani and Segala [6] reduce the problem of 
deciding strong probabilistic bisimulation to LP [21] problems. Zhang and Her- 
manns [31] develop algorithms with improved time complexity for probabilistic 
simulations, following [4,6]. A space efficient probabilistic simulation algorithm 
is proposed by Zhang [30] using the techniques proposed in [14,27]. 

Studies on stochastic games have actually been carried out since as early as 
the 1950s [24], and a rich literature has developed in recent years (e.g. see [11, 10, 
12,7]). One existing approach called game metrics [13] defines approximation- 
based simulation relations, with a kernel simulation characterising the logic quan- 
titative ^-calculus (g/z) [10], an extension of modal /z-calculus [19] where each 
state is assigned a quantitative value in [0, 1] for every formula. However, so 
far the best solutions in the literature on approximating the simulation as de- 
fined in the metrics for concurrent games potentially take exponential time [8]. 
Although PA-simulation is strictly stronger than the kernel simulation relation 
of the game metrics in [13], the algorithm presented in the paper has a more 
tractable complexity result, and we believe that it will benefit the abstraction 
or refinement based techniques for verifying game-based properties. 

Structure of the paper. Sect. 2 defines basic notions that are used in the technical 
part. In Sect. 3 we propose a solution of calculating largest PA-simulation in 
finite PGSs, based on GCPP. The algorithms on PA-simulation is presented in 
Sect. 4. We conclude the paper in Sect. 5. 
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2 Preliminaries 

Probabilistic game structures are defined in terms of discrete probabilistic dis- 
tributions. A discrete probabilistic distribution A over a finite set S is a function 
of type S — > [0, 1], where X^eS ^( s ) = 1- We write V(S) for the set of all such 
distributions on a fixed S. For a set T C S, define A(T) = J2 s et ^K s )- Given a 
finite index set /, a list of distributions (Ai) ieI and a list of probabilities (pi)i e i 
where, for all i e I, pi € [0,1] and ^2 ieI Pi = 1, ^ieiPi^ * s obviously also 
a distribution. For s e S, s is called a pom£ for Dirac) distribution satisfying 
s(s) = 1 and s(t) = for all t ^ s. Given A e T^iS), we define as the set 
{s e S | /4(s) > 0}, which is the support of A. 

In this paper we assume a set of two players {I, II } (though our results can 
be extended to handle a finite set of players as in the standard game structure 
and ATL semantics [2]), and Prop a finite set of propositions. 

Definition 1. A probabilistic game structure Q is a tuple (S, s ,C, Act, 5), where 

— S is a finite set of states, with s the initial state; 

— C : S — > 2 Prop is the labelling function which assigns to each state s G S a 
set of propositions that are true in s; 

— Act — Acti x Acta is a finite set of joint actions, where Acti and Acta are, 
respectively, the sets of actions for players I and H; 

— 5 : S x Act — > V{S) is a transition function. 

If in state s player I performs action a\ and player II performs action ai then 
S(s, {a\,a2)) is the distribution for the next states. During each step the players 
choose their next moves simultaneously. We define a mixed action of player I 
(H) as a distribution over Acti (Actjx), and write II 1 (TTn) for the set of mixed 
actions of player I (n). 4 In particular, a is a deterministic mixed action which 
always chooses a. We lift the transition function <5 to handle mixed actions. Given 
71"! e IIj and 7r 2 e 77 n , for all s,t £ S, we have 

5(s, (7ri,7T 2 ))(i) = ^2 TTi(ai) • 7r 2 (a 2 ) • <5(s, (ai,a 2 ))(t) 

ai£Acti,a2£Actji 

Example 1. Assume Prop = {p}. A simple PGS with the initial state sq in Fig. 1 
can be defined as Q = (S, so, £, Act, 5), where 

- S = {s Q ,si,s 2 }; 

- C(s ) - £(si) = and C(s 2 ) = {p}; 

- Act T = Act-a = {0, 1}; 

- <5(s ,(0,0}) - 5(a ,<0,l» = A with A( Sl ) - A(s 2 ) - \ and tf(a ,<l,0» = 
6(a , (1, 1}) = A' with A'( Sl ) = \, A'(s 2 ) = |; 

4 Note II 1 is equivalent to V(Act 1 ), though we choose a different symbol because the 
origin of a mixed action is a simplified mixed strategy of player I which has type 
S + — > T>(Acti). A mixed action only considers player I's current step. 
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Fig. 1. A probabilistic game structure. 



— S(si,a) = Si for i € {1,2} and a <E Act (s\ and s 2 are absorbing states). 

Simulation relations in probabilistic systems require a definition of lifting [17], 
which extends the relations to the domain of distributions. 5 Let S, T be two sets 
andKCSxTbea relation, then K C V(S) x V(T) is a lifted relation defined 
by A 1Z if there exists a weight function w : S x T — > [0, 1] such that 

— J2teT w ( s >t) = ^( s ) f° r all seS, 

- E sG s W U t) = for a11 * e t, 

- sft t for all s e S and f e T with w(s, t) > 0. 

The intuition behind the lifting is that each state in the support of one distri- 
bution may correspond to a number of states in the support of the other distri- 
bution, and vice versa. The example in Fig. 2 is taken from [22] to show how to 
lift one relation. We have two set of states S — {si, s 2 } and T = {ti, i 2 , t^}, and 
K = {(«i,ti), (*i,t 2 ), (s 2 ,i 2 ), (s 2 ,t 3 )}- We have ATZ0, where A( Sl ) = A(s 2 ) = 
| and 0(t\) — 0{t 2 ) — 0(fz) = |. To check this, we define a weight function 
w by: w(s\,ti) — |, w(si,t 2 ) — g tu(s2,i2) = g, and w(s2,^3) = §■ The dotted 
lines indicate the allocation of weights required to relate A to via 1Z. By lifting 
in this way, we are able to extend the notion of alternating simulation [3] to a 
probabilistic setting. 

Definition 2. Given a PGS (S, So, C, Act, S), a probabilistic alternating I-simulation 
(PA- 1- simulation) is a relation C C S x S such that if s Qt, then 



— for all 7Ti G 77j, there exists Tr[ £ Tlj, such that for all ir' 2 G II n, there exists 
ir 2 G IIu, such thatS(s, {m, 7r 2 )) cl(t, (tti,^)). 

If s PA-I-simulates t and i PA-I-simulates s, we say s and t are PA- 1 -simulation 
equivalent.® 

PA- I-simulation has been shown to preserve a fragment of PATL which covers 
the ability of player I to enforce certain temporal requirements [29] . For example, 

5 In a probabilistic system without explicit user interactions, state s is simulated by 
state t if for every s A Ai there exists t A A2 such that A\ is simulated by A%. 

6 Alternating simulations and equivalences are for player I unless stated otherwise. 



- C{s) = C{t), 
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Fig. 2. An example showing how to lift one relation. 



if in state s player I can enforce reaching some states satisfying p within 5 
transition steps and with probability at least |, written s \= ((l))-? ()- 5 p, then 
for every state t that simulates s with respect to I, i.e., s C t by some PA-I- 
simulation 'C', we also have t \= ((l))-2<>- 5 p. 

General Coarsest Partition Problem 

The general coarsest partition problem (GCPP) provides a characterisation of 
(non-probabilistic) simulation in finite state transition systems [14]. Informally, 
in this approach, states that are (non-probabilistic) simulation equivalent are 
grouped into the same block, and all such blocks form a partition over the (fi- 
nite) state space. Based on the partition, blocks are further related by a partial 
order ^, so that if P ^ Q, then every state in block P is simulated by every 
state in block Q. The GCPP is to find, for a given PGS, the smallest such set 
of blocks. In the literature such a methodology yields space efficient algorithms 
for computing the largest (non-probabilistic) simulation relation in a finite sys- 
tem [14, 27]. Similar methods have been adopted and developed to compute the 
largest simulation relations in the model of probabilistic automata [30] . 

We briefly review the basic notions that are required to present the GCPP 
problem. A partition over a set S, is a collection S C V(S) satisfying (1) (J S = S 
and (2) P n Q = for all distinct blocks P,Q e S. Given s e S, write [s] s for 
the block in partition S that contains s. A partition Si is finer than S 2 , written 
Si < S 2 , if for all P e E\ there exists Q e S 2 such that P C Q. 

Given a set S, a partition pair over S is (S, where S is a partition over 
S and ^ C S x S is a partial order. Write Part(S) for the set of partition 
pairs on S. If Y < S and ^ is a relation on S, then ^ (Y) = {(P, Q) \ P, Q e 
Y, BP', Q' e S, P C P', Q C Q', P> < Q'} is the relation on T induced by < Let 
(Si, ^1) and (S 2 , ^2) be partition orders, write ^1) < (S 2 , ^2) if £1 < £2, 
and ^iC^ 2 (Si). Define a relation S x 5 as determined by a partition 

pair (S, ^) by s E(i;,-<) t iff [s]^ X [t]^. 

Let — >C S x S be a (transition) relation and £ : 5 — > 2 Prop a labelling 
function, then a relation C is a simulation on S if for all s, t € 5 with s C t, we 
have (1) £(s) = C(t) and (2) s -> s' implies f -> t' and s' C i'. Let (S, ±) be a 
partition pair on S, then it is stable with respect to — > if for all P,Q e £ with 
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P ^ Q and s e P such that s -> s' with s' e P' e then there exists Q' £ £ 
such that for all t £ Q, there exists £' € Q' such that i — > i'. The following result 
is essential to the GCPP approach, as we derive the largest simulation relation 
by computing the coarsest partition pair over a finite state space. 7 

Proposition 1. [14,27] Let (£, ^) be a partition pair, then it is stable with 
respect to — > iff the induced relation E(i;,^) is a simulation (with respect to — >J. 

Given a transition relation on a state space there exists a unique largest sim- 
ulation relation. Thus, solutions to GCPP provide the coarsest stable partition 
pairs, and they have been proved to characterise the largest simulation relations 
in non-probabilistic systems [14,27]. 

3 Solving GCPP in Probabilistic Game Structures 

In this section we extend the GCPP framework to characterise PA-simulations 
in PGSs. Given a PGS Q = (S,So,£, Act,S), a partition pair over Q is (£, ^) 
where £ is a partition over S. Write Part(Q) for the set of all partition pairs 
over S. We show how to compute the coarsest partition pair and prove that it 
characterises the largest PA-simulation for a given player. 

Since in probabilistic systems transitions go from states to distributions over 
states, we first present a probabilistic version of stability, as per [30]. Let — >C 
S x V(S) be a probabilistic (transition) relation. For a distribution A e V{S) 
and £ a partition, write As as a distribution on £ defined by As(P) = A(P) 
for all P <G £. Let (£, be a partition pair, it is stable with respect to the 
relation — >, if for all P,Q £ £ with P < Q and s € P such that s — »■ A, then for 
alii e Q there exists t — > such that As ;< 0s- 

Another obstacle in characterising PA-simulation is that the concerned player 
can only partially determine a transition. That is, after player I performs an 
action on a state, the exact future distribution on next states depends on an 
action from player H. Therefore, we need to (again) lift the stability condition 
for PA-I-simulation from distributions to sets of distributions. 

Let < C S x S be a partial order on a set S, define <s m C V(S) x V(S), by 
P <sm Q if for all i € Q there exists s e P such that s < t. In the literature this 
definition is known as a 'Smyth order' [25]. In a PGS, we 'curry' the transition 
function by defining 5(s,iri) — {S(s, (ni,Tr 2 )) | n 2 € i7 n }, which is the set of 
distributions that are possible if player I takes a mixed action m e LTj on s e S. 

Definition 3. (lifted stability) Let (£, be a partition pair on S in a PGS, it 
is stable with respect to player I's choice, if for all n e TTj, P,Q e £ with P <Q 
and s € P, there exists ir' € TTj such that S(s,tt)s dism^{t^')s for all t e Q. 

Intuitively, the Smyth order captures the way of behavioral simulation. That is, 
if 5(t, 7r') is at least as restrictive as 5(s,ir), then whatever player I is able to 

7 We choose the word coarsest for partition pairs to make it consistent with the stan- 
dard term GCPP, and it is clear in the context that coarsest carries the same meaning 
as largest with respect to the order < defined on partition pairs. 
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enforce by performing it in s, he can also enforce it by performing it' in t, as 
player H has fewer choices in 5(t, n') than in 5(s, n). At this point, for the sake of 
readability, if it is clear from the context, we write W for We as the distribution 
W mapped onto partition S. 

For simulation relations, it is also required that the related states agree on 
their labelling. Define S as the labelling partition satisfying for all s,t £ S, 
C(s) = C{t) iff [s]s = [t]s Q - Write Part°(G) C Part(Q) for the set of partition 
pairs (S, if!) satisfying (S, ^) < (S , Id), where Id is the identity relation. 

Lemma 1. For all (S, ^) e Part°(G), if (S, if?) is a stable partition pair with 
respect to player I's choice then E(i;,-<) is a PA- 1- simulation. 

Proof. Straightforward by Definition 3. 

Obviously every PA-I-simulation is contained in the relation induced by 
(S , Id), and moreover, the above lemma asserts that every stable partition pair 
smaller than (So, Id) is a PA-I-simulation. In the following, we try to compute 
the coarsest partition pair by refining (So, Id) until it stabilises. The resulting 
stable partition pair can be proved to characterise the largest PA-I-simulation 
on the state space S as required. 

We say t simulates s with respect to player- I's choice on a partition pair 
(S, <) if for all it e IIj, there exists n' G II T such that 5(s, ir) 3sm S(t, n'). For 
better readability, sometimes we also say t simulates s on (S, X) if it is clear 
from the context, and write s C*^ t. Note it is straightforward to show that 
— (e x) ^ s a transitive relation, by definition of ^g m . Let (i?i,^i) < (S 2l ^< 2 ), 
we say (Si, ^1) is stable on (S 2 , ^2), if for all P,Q e £i with P -<i Q, s e P 
and t e Q, t simulates s on (S 2 , ^2)- 

Definition 4. Define an operator p : Part(Q) — > Part(Q), such that p((S,^)) 
is the largest partition pair (S 1 , X') < (S, <) that is stable on (S, X). 

The operator p has the following properties. 

Lemma 2. p is well defined on Part(Q). 

Proof. We show that given a partition pair (S, -<) on S, p((S, X)) is a unique 
partition pair. Let P G S. Define < P C P x P by s < P t if s _^ t. Then 
<p is a preorder on P, from which we define a partition pair (Sp, X P ) where 
S P = {{t e P I s < P t A t < P s} I s e P} and X x < P X 2 if there exist s e X x 
and t e X 2 such that s < P t. Define p((i7, X)) = (S' , <') with Z" = (jp e s S p 
and X'= (X \Id)(S') U Upei; ^p- For thc definition of X', the first part of the 
union (X \Id)(S') is the relation on S' as induced from the nonrcflcxive part of 
X, and in the second part each <p gives a new relation generated inside block 
P which is stable on (S, X). Note that each X P is acyclic, and thus a partial 
order on Sp. This implies that -<' is a partial order on S' . 

We show that (S' , <') is indeed the largest such partition pair. Suppose there 
exists (S", ■<") such that (S" , <") < (S, ■<) and it is stable on (S, X), we show 
that (S",<") < (S',<'). 
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- Let P £ E" and s £ P, then there exists P' £ E' such that s £ P' . First we 
have P C [s] s by E" < Z\ For all t £ P, we have s C*^ _^ f and f C* £ _^ s, 
by P stable on (E, ^). By definition we have s <[ S ] E * an d * — s ' ano - 
thus i e P'. Therefore, P C P'. This proves Z"' < E'. 

- Let P,Q £ E" and P ^" Q. Since Z"' < Z", there exist P',Q' £ E' such 
that P C P' and Q C Q'. We need to show that P' <?'• Taking si e P' 
and S2 € Q', we show that Si C*^ _^ S2- Let fa £ P and £2 € Q, we have 
fa !=(£■-<) *2- Also within [si]s we have Si ^[ Sl ] s fa, and within [s2]j; we 
have t 2 ^[s 2 ] s s 2- As both and zf?[ S2 ] E are contained in Ct^^., We 
apply transitivity to get Si C*^ _^ s 2 - Therefore, P' ^' Q' . This shows that 
(P,<3) G =<' (17"). and thus (£")■ 

□ 

The following lemma is used in the proof of Lemma 4. 

Lemma 3. If (E\,^i) < (E 2 , ^2) fftere are distributions A, A' satisfying 
=<7 ^ , <ftcn Z^ 2 ^2 • 

Proof, (sketch) By reusing the same weight function for ^1 on the partition E\ 
for -<2 on the coarser partition E 2 . □ 

Lemma 4. p is monotonia on (Part°(Q), <). 

Proof. Let (E 1 ,^ 1 ) < {E 2 ,< 2 ), {E[, ±[) - p((2i,±i)) and (Z 2 , <' 2 ) = p((E 2 , < 2 
)). Wc show that (E[, <[) < {E' 2 ,<' 2 ). 

We first prove that (E^,^) is stable on (E 2 ,< 2 ). Let P, Q £ E[ such that 
P ^'1 Q, then for all s £ P, t e Q and tt £ II 1 , there exists n' £ IJj such 
that S(s,tt) <\ 6(t,n'). Then by Lemma 3, we also have S(s, n) -< 2 S(t, n'). By 
definition of p, we have that the partition pair {E 2 ,<' 2 ) is the unique largest 
partition pair that is stable on (E 2 ,< 2 ). As (E[,^' 1 ) is stable on (£2,^2), it 
must be the case that (E[, ^[) < (E' 2 , <' 2 ). □ 

Lemma 1 ensures that for all (E, ^) £ Part (Q), E(.e,-<) is a PA-I-simulation 
if p((17, if!)) = ^), i.e., (Z, d) is a fixpoint of /?. However, we still need 
to find the largest PA-I-simulation. The following result indicates that if S is 
finite, the coarsest stable partition pair achieved by repetitively applying p on 
(Eg, Id) indeed yields the largest PA-I-simulation. 8 Define p°(X) = X and 
p n+1 {X) = p(p n (X)) for partition pairs X. 

Theorem 1. Let (E,^) = f] ieN p l ((E , Id)), then E(i;,^) is the largest PA-I- 
simulation on Q . 

Proof, (sketch) Let C+ be the largest PA-I-simulation on Q. Define a set E + = 
{{t £ S I s C+ t A t C+ s} I s £ S}. Since C+ is the largest PA-I-simulation, it 

8 The following proof resembles the classical paradigm of finding the least fixpoint in 
an o>-chain of a complete partial order by treating (So, Id) as _L. However, here we 
also need that fixpoint to represent the largest PA-I-simulation. 
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can be shown that C+ is reflexive, symmetric and transitive within each block 
P E S + . Moreover, we define a relation < + by P < + Q if there exists s e P 
and t £ Q such that s C+ t, and it can be shown that :<+ is a partial order on 
S + . Then (S + , < + ) forms a partition pair on Q, and furthermore, it is stable, 
and we also have < (^o, Id). 

We apply p on both sides. By Lemma 4 (monotonicity) , and (S + , ^ + ) being 
stable, we have {£+,<+) = p i {{S+,< + )) < p^X^Id)) for all i € N. As 
Part(g) is finite, there exists j e N, such that p>{(E ,Id)) = f>> +1 ((Eo,Id)). 
Therefore, p 3 ((S , Id)) is a stable partition pair, and Qpius ,id)) i s a PA-I- 
simulation by Lemma 1. Straightforwardly we have C+CC^-^^ |Id ^. Since C+ 
is the largest PA-I-simulation by assumption, we have Q + =Qpj((E 0t i&)), an d the 
result directly follows. □ 



4 A Decision Procedure for PA-I-Simulation 

Efficient algorithms for simulation in the non-probabilistic setting sometimes 
apply predecessor based methods [16, 14] for splitting blocks and refining parti- 
tions. This method can no longer be applied for simulations in the probabilistic 
setting, as the transition functions now map a state to a state distribution rather 
than a single state, and simulation relation needs to be lifted to handle distri- 
butions. The algorithms in [31,30] follow the approaches in [4] by reducing the 
problem of deciding a weight function on lifted relations to checking the value 
of a maximal flow problem. This method, however, does not apply to combined 
transitions, where a more general solution is required. Algorithms for deciding 
probabilistic bisimulations [6] reduce the problem on checking weight functions 
with combined choices to solutions in linear programming (LP), which are known 
to be decidable in polynomial time [18]. 9 

Simulation relations are characterised by partition pairs in the solutions to 
the GCPP. We propose the following characterisation of lifting in order to handle 
the partial order relation on partitions. Let S be a finite set and ^ a partial 
order on S. Define [s\^ — {t E S | s ^ t}, which is called the up-closure of 
s. The following lemma reduces the problems of finding a weight function for 
two distributions on a partition pair to comparing weights of each up-closed 
block, and the latter problem can be easily encoded in LP when checking PA-I- 
simulation on a given partition pair between two states (as shown in Lemma 9). 

Lemma 5. Let S be a set with a partial order -< C S x S and A\,Ai € T> (S), 
then Ai ^ A 2 iff we have Z\i([sJ^) < Z\ 2 ([sJ^) for all s e S. 

Proof. — For 'only if, suppose A\ -< A 2 , then there exists a weight function w 
to establish the lifting. For each s e S, we have A\ ( |_sj ^ ) = J2te [s] -< (*) = 
EteWi Et'es^M') = EteWi ^'e[»J^ w(t,t'). The last equality is due 
to w(t,t') > implies t ^ t', we can 'purge' those states t that are not 

9 The maximal flow problem is a special instance of an LP problem, which can be 
solved more efficiently [1]. 
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— For 'if, suppose Zii([sjx) < ^2 ( L S J ^ ) f° r an s E S, we establish a weight 
function for Ai<A 2 . We recursively show that w is definable on each seS 
by traversing the states by the (topological) order of ^. (One may treat 
w as of type S — >• S — >• [0,1].) If [sjx = {s}, i.e., s is a top state, then 
Z\i(s) < ^ 2 (s), and w is definable on s, e.g. by letting w(s,s) — A\(s) 
Otherwise, we have [s\ x = {s} U Usxt L*J ^ where s -< t denotes s <t, s ^ t 
and there does not exist r such that s < r < t. Since Zii([tjx) < ^2 ( L^J ^ ) 
for all t satisfying s ~< t , we write f (i) for Z\ 2 ( [t\ -< ) — Zii ( [ij -< ) , and we have 
^i(s) < ^2(5) + J2 s -<t t (*)■ As the 'remaining weight' is at least as large 
as Ai(s), one can easily show that w is definable on s. □ 

When deciding whether s is able to simulate t with respect to I's choice on 
a certain partition pair, we need to examine potentially infinitely many mixed 
actions in 77 x . This problem can be moderated by the following observations. 
First we show that for s to be simulated by t, it is only required to check all 
deterministic choices of player I on s. 

Lemma 6. Let (S,^) be a partition pair, then t simulates s on if for 

all a e Acti, there exists it e 77 j such that S(s,a) <sm S(t,n). 

The next lemma states that for checking a Smyth order 6(s,ir) ^Sm5{t,ir'), 
it suffices to focus on player H's deterministic choices in 6(t,w'), since all prob- 
abilistic choices can be represented as interpolations from deterministic choices. 



Lemma 7. 5(s, w) -<sm S(t,n') if for all a e Acta, there exists n" e 77u such 
that 8(s, {■K,'w"))<8{t, (tt',5)). 



Combining the above two lemmas, we have the following. 

Lemma 8. Let (E, ■<) be a partition pair, then t simulates s with respect to 
player-I's choice on (S,^.) if for all a\ £ Actj, there exists tt\ € 77j such that 
for all a 2 € Act n , there exists n 2 € LI n such that S(s, (ai,n 2 )) ^ 8(t, (771,02)). 

The following lemma states how to check if the action a can be followed by 
a mixed action from 77];. 

Lemma 9. Given a partition pair (S, ^), two states s,t € S and a € Actj, there 
exists 7T e TTj such that 5(s, a) ^sm S(t, n) iff the following LP has a solution: 
Let Act! = {ai, a 2 , . . . , ae} and Acta = {61, b 2 , ■ ■ ■ , b m } 



i 




(1) 



Vi = 1,2 



...,£: < ^ < 1 



(2) 




(3) 



fe=i 
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Vj,fc = l,2,...,m:0<#,-, fc <l 



(4) 



MB e £ : j = 1,2,..., m : 



m £ 



^2Pj,k ■ S(s,a,b k )([B\-<) < • S(t,ai,bj)([B\^) 



(5) 




Here a\, a.i, . . . , ag are used to 'guess' a mixed action from player I, as con- 
strained in Eq. 1 and Eq. 2. To establish the Smyth order ^5 m , by Lemma 8, 
for every player II action bj with j — 1,2, ... ,m, we 'guess' a mixed action 
from Actji represented by /3j t i,/3j t 2 ■ ■ ■ ,{3j,m, as constrained in Eq. 3 and Eq. 4. 
Then for each block B in £, the established distributions need to satisfy the 
lifted relation <, which is characterised by the inequalities on the up-closure of 
B with respect to the order by Lemma 5. 

We define a predicate CanFollow such that CanFollow((Z', <),s,t,a) decides 
whether there exists a mixed action of player I from t which simulates action 
a e Acti from s on the partition pair (£, <). CanFollow establishes an LP prob- 
lem from its parameters (see Lemma 9). We further define a predicate CanSim 
which decides whether a state simulates another with respect to player I's 
choice on (£, for all actions in Acti, l - c -, CanSim((I7, <), s, t) returns true 
if CanFollow((i7, s, t, a) returns true for all a e Actj. 



Algorithm 1 Refining a block to make it stable on a partition pair 
INPUT: a partition pair (E, X), a block B G E 
OUTPUT: a partition pair [E b ,<b) on B 
function Split ((X 1 , <), B) 

E B := {{s} |s€B}; <b~ {(s,s) | s G B}; E' := 0; <' := 
while E B + EN < B ^<' do 
E' := Eb\ di '- = diB 
for each distinct B\,B2 G Eb do 
pick any si € B\ and s 2 G B 2 

if (CanSim((X', si, s 2 ) A CanSim((X', <), s 2 , si)) then 
E B ~ E B \ {B 1 ,B 2 } U {Si U B 2 } 

<b~<b U {{X, Si U B 2 ) I X G E : (X, Bj) e< B V (X, B 2 ) £<b} 



U{(B!UB 2 ,X) I X G E : {B U X) Gr<s V (B 2 ,X) Gr<s} 
\{(Bi,X), (X,Bi) ! X G E : [B l ,X),{X,B i ) Gr<s A i G {1,2}} 
else if (CanSim((X', -<), si, s 2 )) then 

<B~<B U{(B 2 ,Bi)} 
else if (CanSim((X', -<), s 2 , si)) then 
<b~ <b U{(Bi,B 2 )} 



endfor 
endwhile 
return (Eb, ^s) 



Algorithm 1 defines a function Split which refines a block B e £ into a par- 
tition pair corresponding the maximal simulation that is stable on (£, ^). It 
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starts with the finest partition and the identity relation (as the final relation 
is reflexive). For each pair of blocks in the partition, we check if they can sim- 
ulate each other by picking up a state from each block. If they are simulation 
equivalent on (E, ^) then we merge the two blocks as well as all incoming and 
outgoing relation in the current partial order. If only one simulates the other we 
add an appropriate pair into the current ordering. This continues until the par- 
tition pair stablises. 



Algorithm 2 Computing the Generalised Coarsest Partition Pair 
INPUT: a probabilistic game structure Q = (S, sq,C, Act, S) 
OUTPUT: a partition pair (E, <) on S 
function GCPP (Q) 

E := {{t | C(t) = C(s)} | a G S}; {(B, B) \ B e E} 
E' := 0; <':= 
while E E'V <j=<' do 
E' := E; ^':=^ 
for each B £ E do 

(E B ,< B ) :=Sp\lt((E',<'),B) 
E :— E\ {B} U E B 

< ■= < U <B 

U {(B' , X) | X G E : B' G E B : (B, X) G r<} 
U {(X, B') | X G E : B' G E B : (X, B) G r<} 
\{(B, X), (X, B) | X G E : (X, B), (B, X) G r<} 

endfor 
endwhile 
return (E, ■<) 



Algorithm 2 is based on the functionality of Split in Algorithm 1. Start- 
ing from the partition (E ,Id), which is identified as ({{t \ C(t) = C(s)} \ 
s e S},{(B,B) | B e £b})i the algorithm computes a sequence of partition 
pairs (Hi, dii), (^2, ^2) • • • until it stabilises, which is detected by checking the 
condition S 7^ S' V ■ Each time we have (£i+i, r^i+i) < (£i,^i), and 

moreover, (S i+ i, is the maximal partition pair that is stable on (Si, ^). 

The correctness of the algorithm is justified by Theorem 1, which states that it 
converges to the coarsest partition pair that is contained in (E , Id) and returns 
a representation of the largest PA-I-simulation. 

Space complexity. For a PCS (S, sq, L, Act, $), it requires 0(151) to store the 
state space and ©(IS"! 2 • \Act\) for the transition relation, since for each seS 
and (01,02) G Act it requires an array of size ©(IS*!) to store a distribution. 
Recording a partition pair takes 0(\S\ log \S\ + \S\ 2 ) as the first part is needed 
to record for each state which equivalence class in the partition it belongs, and 
the second part is needed for the partial order relation ^ which takes at most 
©(IS*! 2 ). The computation from (£~i,<i) to (£~i+i, r^»+i) can be done in-place 
which only requires additional constant space to track if it has been modified 
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during each iteration. Another extra space-consuming part is for solving LP 
constrains, which we assume has space usage 0(j(N)) where N = 1 + |-i4c£i| + 
|^4c£ n | + |^4c£ n | 2 + \S\ ■ |^4ct n | is the number of linear constraints at most, and 
"f(N) some polynomial. The space complexity roughly sums up to ©(IS*! 2 • \Act\ + 
\S\ log |5| + l(\Act\ 2 + \S\ ■ \Act\)). (The first part C(|5| 2 • \Act\ + \S\ log \S\) for 
the PGS itself can be considered optimal, while the second part depends on the 
efficiency of the LP algorithm being used.) 

Time complexity. The number of variables in the LP problem in Lemma 9 is 
|^4c£i| + |^4cin| 2 , and the number of constraints is bounded by l + |^4c£i| + |j4ctn| + 
|^4c£n| 2 + \S\ ■ \Actn\. The predicate CanSim costs \Acti\ times LP solving. Each 
Split invokes at most \B\ 2 testing of CanSim where B is a block in E. Each 
iteration of GCPP splits all current blocks, and the total number of comparisons 
within each iteration of GCPP is be bounded by IS"! 2 . (However it seems heuristics 
on the existing partition can achieve a speed close to linear in practice by caching 
previous CanSim checks [31].) The number of iterations is bounded by \S\. This 
gives us time complexity which is in the worst case to solve C(| Acfi | • IS*! 3 ) many 
such LP problems, each of which has 0(\S\ ■ \Act\ + \Act\ 2 ) constraints. 

Remark. By removing the interaction between players (i.e., the alternating part), 
our algorithm downgrades to a partition-based algorithm computing the largest 
strong probabilistic simulation relation in probabilistic automata, where com- 
bined transitions are needed. The algorithm of [31] for computing strong proba- 
bilistic simulation has time complexity of solving C(|S'| 2 -m) LP problems, where 
m is the size of the transition relation comparable to COS 1 ) 2 • | ^4ci| ) . They have 
0(| S^ 2 ) constraints for each LP instance. The improvement achieved in our al- 
gorithm is due to the use of partitions in each iteration instead of working on 
the whole relation, which is made possible by applying Lemma 5. 

The space-efficient algorithm [30] for probabilistic simulation (without com- 
bined transitions) has the same space complexity but better time complexity 
than ours, which is due to the reduction to the maximal flow problem. 

5 Conclusion 

We have presented a partition-based algorithm to compute the largest probabilis- 
tic alternating simulation relation in finite probabilistic game structures. To the 
best of our knowledge, our work presents the first polynomial-time algorithm for 
computing a relation in probabilistic systems considering (concurrently) mixed 
choices from players. As aforementioned, PA-simulation is known as stronger 
than the simulation relation characterising quantitative /i-calculus [13], though 
it is still a conservative approximation which has a reasonable complexity to be 
useful in verification of game-based properties. 
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